<!DOCTYPE html>
<!-- Copyright 2017 Yahoo Holdings. Licensed under the terms of the Apache 2.0 license. See LICENSE in the project root. -->
<html lang="en">
<head>
  <meta charset="utf-8" />
  <title>Vespa Test Framework Tutorial</title>
[insert:file:style.inc]
</head>
<body>

<header>
<h1>Vespa Test Framework Tutorial</h1>
</header>

<section>
<div class="page-header">
<h1>Introduction</h1>
</div>

<p>
The Vespa test framework helps you write small applications to test
C++ code. All interaction with the test framework is done with the use
of macros.
</p>

<p>
The minimal test application looks like this:
</p>

[insert:example:minimal/minimal_test.cpp]

<p>
The runnable application itself is called a <strong>test
suite</strong> and inherits its name from the cpp file containing the
TEST_MAIN macro. Each individual verification of some value is called
a <strong>check</strong>. Checks can be put anywhere, but it is highly
recommended that you put them inside <strong>tests</strong>. Tests are
created by a family of macros. Another macro (TEST_RUN_ALL) is used to
execute them.
</p>

<p>
Example with two tests, each containing a single check:
</p>

[insert:example:simple/simple_test.cpp]
</section>


<section>
<div class="page-header">
<h1>Checks</h1>
</div>

<p>
All checks are available in two variants. Those with the
<strong>EXPECT_</strong> prefix allow execution to continue even if a
check fails. Those with the <strong>ASSERT_</strong> prefix will
terminate execution of the current test if it fails.
</p>

[insert:example:checks/checks_test.cpp]

<p>
Checks involving comparison of values typically use == and &lt;
operators to compare values. Also; in order to be part of a comparison
check, the value must support the &lt;&lt; operator to print the value
to a string stream in case the check fails.
</p>
</section>


<section>
<div class="page-header">
<h1>Test Fixtures</h1>
</div>
<p>
Sometimes multiple tests wish to use the same predefined state as a
starting point. This state is called a test fixture. Test fixtures are
untangled from the actual tests and construction/destruction is used
to handle their lifetime. When thinking of what can be used as a
fixture, think of what can be put after <strong>new</strong> to create
an object.
</p>

<ul>
<li>A single test can have multiple test fixtures.</li>
<li>The number of <strong>F</strong>s in the test macro denotes the number of fixtures.</li>
<li>Inside the test, fixtures are available as <strong>f1</strong>, <strong>f2</strong> and so forth.</li>
<li>Test fixtures can be parameterized with constructor parameters.</li>
<li>Checks can be performed inside test fixture constructors.</li>
<li>A test fixture constructor can take other test fixtures as input.</li>
</ul>

[insert:example:fixtures/fixtures_test.cpp]
</section>


<section>
<div class="page-header">
<h1>Multi-Threaded Tests</h1>
</div>
<p>
One of the most novel features of the test framework is the ability to
write multi-threaded tests. Multi-threaded tests are created by using
test macros containing <strong>_MT</strong> and supplying a thread
count. All threads will execute the block of code encapsulated by the
test. The test fixtures are shared among all threads. In addition,
each thread has a variable named <strong>num_threads</strong>
containing the total number of threads running the test and a variable
named <strong>thread_id</strong> identifying the thread.
</p>

<p>
The <strong>TEST_BARRIER()</strong> macro can be used inside the test
to synchronize the threads. The macro will block execution of each
thread invoking it until all threads have invoked it.
</p>

[insert:example:threads/threads_test.cpp]
</section>


<section>
<div class="page-header">
<h1>Real World Examples</h1>
</div>
[insert:example:../box/box_test.cpp]
[insert:example:../barrier/barrier_test.cpp]
[insert:example:../dual_merge_director/dual_merge_director_test.cpp]
</section>


<section>
<div class="page-header">
<h1>Macro Summary</h1>
</div>

<h2>Overall Execution Macros</h2>
<ul>
<li><code>TEST_MAIN()</code></li>
<li><code>TEST_RUN_ALL()</code></li>
</ul>

<h2>Test Creation Macros</h2>
<ul>
<li><code>TEST(name)</code></li>
<li><code>TEST_F(name, fixture)</code></li>
<li><code>TEST_FF(name, fixture1, fixture2)</code></li>
<li><code>TEST_FFF(name, fixture1, fixture2, fixture3)</code></li>
</ul>

<ul>
<li><code>TEST_MT(name, threads)</code></li>
<li><code>TEST_MT_F(name, threads, fixture)</code></li>
<li><code>TEST_MT_FF(name, threads, fixture1, fixture2)</code></li>
<li><code>TEST_MT_FFF(name, threads, fixture1, fixture2, fixture3)</code></li>
</ul>

<ul>
<li><code>IGNORE_TEST(name)</code></li>
<li><code>IGNORE_TEST_F(name, fixture)</code></li>
<li><code>IGNORE_TEST_FF(name, fixture1, fixture2)</code></li>
<li><code>IGNORE_TEST_FFF(name, fixture1, fixture2, fixture3)</code></li>
</ul>

<ul>
<li><code>IGNORE_TEST_MT(name, threads)</code></li>
<li><code>IGNORE_TEST_MT_F(name, threads, fixture)</code></li>
<li><code>IGNORE_TEST_MT_FF(name, threads, fixture1, fixture2)</code></li>
<li><code>IGNORE_TEST_MT_FFF(name, threads, fixture1, fixture2, fixture3)</code></li>
</ul>

<h2>Check Macros</h2>
<ul>
<li><code>ASSERT_TRUE(rc)</code></li>
<li><code>ASSERT_FALSE(rc)</code></li>
<li><code>ASSERT_EQUAL(a, b)</code></li>
<li><code>ASSERT_NOT_EQUAL(a, b)</code></li>
<li><code>ASSERT_APPROX(a, b, eps)</code></li>
<li><code>ASSERT_NOT_APPROX(a, b, eps)</code></li>
<li><code>ASSERT_LESS(a, b)</code></li>
<li><code>ASSERT_LESS_EQUAL(a, b)</code></li>
<li><code>ASSERT_GREATER(a, b)</code></li>
<li><code>ASSERT_GREATER_EQUAL(a, b)</code></li>
<li><code>ASSERT_EXCEPTION(statement, exception_type, msg_substr)</code></li>
<li><code>TEST_FATAL(msg)</code></li>
</ul>

<ul>
<li><code>EXPECT_TRUE(rc)</code></li>
<li><code>EXPECT_FALSE(rc)</code></li>
<li><code>EXPECT_EQUAL(a, b)</code></li>
<li><code>EXPECT_NOT_EQUAL(a, b)</code></li>
<li><code>EXPECT_APPROX(a, b, eps)</code></li>
<li><code>EXPECT_NOT_APPROX(a, b, eps)</code></li>
<li><code>EXPECT_LESS(a, b)</code></li>
<li><code>EXPECT_LESS_EQUAL(a, b)</code></li>
<li><code>EXPECT_GREATER(a, b)</code></li>
<li><code>EXPECT_GREATER_EQUAL(a, b)</code></li>
<li><code>EXPECT_EXCEPTION(statement, exception_type, msg_substr)</code></li>
<li><code>TEST_ERROR(msg)</code></li>
</ul>

<h2>Thread Macros</h2>
<ul>
<li><code>TEST_BARRIER()</code></li>
</ul>

<h2>State Tracking Macros</h2>
<ul>
<li><code>TEST_DO(doit)</code></li>
<li><code>TEST_STATE(msg)</code></li>
</ul>

<h2>Macros of Limited Use</h2>
<ul>
<li><code>TEST_TRACE()</code></li>
<li><code>TEST_FLUSH()</code></li>
<li><code>TEST_THREAD(name)</code></li>
<li><code>TEST_DEBUG(lhsFile, rhsFile)</code></li>
<li><code>TEST_MAIN_WITH_PROCESS_PROXY()</code></li>
</ul>
</section>

<script type="text/javascript">
$(function(){
    window.prettyPrint && prettyPrint();
});

</script>

</body>
</html>
